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(54) Transparent embedment of data in a video signal 



(57) A method and apparatus for transparently em- 
bedding data into a video signal modulates a particular 
carrier frequency with a randomized pattern represent- 
ing the data. The randomized pattern may be a combi- 
nation of the data with the output from a pseudo-random 
binary sequence generator, or the data may be convert- 
ed first into a bi-orthogonal pattern selected from among 
a plurality of such patterns and then randomized. The 
resulting modulated carrier frequency is then amplitude 
modulated based upon a human visual model so that 
the modulated carrier frequency amplitude is greater in 



areas of high image complexity and lower in areas of 
low image complexity. The subliminal data represented 
by the modulated carrier frequency is added to the video 
signal. At a decoder the data is recovered using appro- 
priate correlation techniques by first filtering the video 
signal to increase the subliminal data component rela- 
tive to the image component, then demodulating the da- 
ta component to produce the data pattern, and then cor- 
relating the data pattern with the possible data patterns 
to identify the particular pattern and associated data 
word. 
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Description 

CROSS-REFERENCE TO RELATED APPLICATIONS 

Not applicable 

STATEMENT REGARDING FEDERALLY 
SPONSORED RESEARCH OR DEVELOPMENT 

Not applicable 

BACKGROUND OF THE INVENTION 

The present invention relates to data transmission, 
and more particularly to a method and apparatus for 
transparently embedding data within a video signal in 
order to indicate authentication, program ownership 
and/or reception-verification of the video signal. 

Television signals are usually copyrighted or other- 
wise proprietary to the originator and, in the case of tel- 
evision network distribution, are distributed to affiliate 
stations for re-broadcast. Unauthorized re-broadcast is 
difficult to detect since it is often difficult to determine 
the creator or originator of the material from the material 
itself. This is particularly true for short sequences or 
when the video image has been cropped or modified to 
mask any identification logos that may have been ex- 
plicitly burned into the video image. 

Another common problem with television and video 
distribution, which is becoming more severe, is the syn- 
chronization of an aural or audio component that is to 
be distributed with the video signal. Modem digital signal 
processing techniques requiring large buffering, such as 
MPEG compression, add latency to the video distribu- 
tion and, since the audio may be distributed or proc- 
essed separately, an error in audio to video "lip-sync" or 
in sound to action often occurs. Sometimes this latency 
is variable, requiring continual re-synchronization. In ad- 
dition to audio to video synchronization problems, the 
audio signal may be distorted or missing entirely. By em- 
bedding, for example, the audio envelope as data within 
the video signal, it Is also possible to detect and com- 
pare the received audio with the original which was cod- 
ed and embedded as data within the video signal as a 
quality measure as well as video to audio delay. 

Another problem is with authentication of video sig- 
nals representing visual images, possibly created by a 
computer, where it is desirable to detect a forgery or im- 
poster signal. In this case the video signal may be rep- 
licated to such a degree that it is difficult to detect it from 
the authentic video sequence. Also it is sometimes de- 
sirable to detect the beginning and/or ending of a par- 
ticular sub-sequence of a video signal, such as a motion 
test sequence for in-service video quality assessment 
of video processed by MPEG compression. It is possible 
to devise a remote receiver to capture a special test se- 
quence segment of a distributed video signal and com- 
pare that segment to an undistorted, stored version to 



assess the quality of the video. Although it is possible 
to uniquely and reliably detect when the sequence oc- 
curs so that it, and only it, is captured, this generally re- 
quires the advance processing of a signature vector, 
$ transmission of that vector through a separate channel 
and a correlation means at the receiver. By transparent- 
ly embedding data into the video a predetermined start 
and end code may be detected by the receiver Without 
any preloading of signature vectors and signature vector 
10 preprocessing. 

Published Canadian Patent Application No. 
2,174,413 (A1) by Geoffrey B. Rhoads entitled "Identi- 
fication/Authentication Coding Method and Apparatus" 
discusses techniques for providing authentication of im- 
15 age signals. In Rhoads an imperceptible N-bit identifi- 
cation code is embedded throughout an image with a 
small noise pattern in a coded fashion. In particular bits 
of a binary identification code are referenced sequen- 
tially to add up to N independent, noise-like patterns to 
the original image signal. The detection of these pat- 
terns is done by N sequential correlations with stored 
replicas of each pattern. This may also be done simul- 
taneously by N correlators, as is well described in the 
public domain as a "correlation receiver." Rhoads fur- 
ther discloses adding or subtracting exactly N independ- 
ent, noise-like images to improve detection and/or en- 
coded image quality. This later modification, referred to 
by Rhoads as "true polarity", is well described in the pub- 
lic domain as "bi-orthogonal signaling" for a correlation 
receiver allowing 2*N composite symbols (patterns) to 
be created by adding or subtracting N bi-orthogonal 
symbols (patterns). A disadvantage of both of Roads' 
methods is that each of the N noise-like patterns, which 
are added or subtracted to form one of the 2*N possible 
composite patterns, needs to be properly scaled and de- 
signed to minimize image degradation by the addition 
of the composite pattern to the image. A further disad- 
vantage is that the original unencoded image, as well 
as the N patterns, need to be stored in the receiver so 
that the unencoded image may be subtracted from the 
encoded image for detection. 

U.S. Patent No. 4,969,041 issued November 6, 
1990 to William J. O'Grady and Robert J. Dubner enti- 
tled "Embedment of Data in a Video Signal" discloses 
adding one of a plurality of tow-level waveforms to the 
video signal, with the level of the low level waveform be- 
ing below the noise level of the video signal, for authen- 
tication purposes or for transmitting information. Each 
low-level waveform corresponds to a particular data 
word being embedded. At the receiving end the video 
signal is correlated with an identical set of low-level 
waveforms to produce a set of correlation coefficients - 
the highest correlation coefficient indicating the pres- 
ence of a particular one of the low-level waveforms 
which is then converted into the corresponding data 
word. The Rhoads composite pattern consists of the 
summation of up to N independent patterns, which 
would be the same as O'Grady/Dubner when N=1 since 
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only one pattern is sent at a time. For an N-bit data word 
O'Grady/Dubner implies the need for storage of 2*N pat- 
terns rather than the N patterns of Rhoads. But since 
2*N patterns may be generated by summing N bi-or- 
thogonal sub-patterns, the effect is the same and only 
N patterns need to be stored in a manner identical to 
Rhoads. In both of these patents the symbol rate is one 
pattern per picture or field. 

Some other drawbacks of O'Grady/Dubner are that 
it limits the degree to which a pattern or sequence of 
patterns can be hidden by not fully exploiting the limita- 
tions of the human psyche-visual process, such as spa- 
tial masking; it does not fully exploit the available signal 
to noise ratio since the correlation output is unipolar; and 
it does not include coding of the data represented by the 
embedded symbols so as to provide temporal redun- 
dancy for error correction over missing video frames. 

What is desired is a method of embedding an un- 
obtrusive data pattern into the video signal that is useful 
in addressing the above-noted problems. 

BRIEF SUMMARY OF THE INVENTION 

Accordingly the present invention provides a meth- 
od and apparatus for transparently embedding data into 
a video signal by modulating a particular carrier frequen- 
cy with a randomized pattern representing the data. The 
randomized pattern may be a combination of the data 
with the output from a pseudo-random binary sequence 
generator, or the data may be converted first into a bi- 
orthogonal pattern selected from among a plurality of 
such patterns and then randomized. The resulting mod- 
ulated carrier frequency is then amplitude modulated 
based upon a human visual model so that the modulated 
carrier frequency amplitude is greater in areas of high 
image complexity and lower in areas of low image com- 
plexity. The subliminal data represented by the modu- 
lated carrier frequency is added to the video signal. At 
a decoder the data is recovered using appropriate cor- 
relation techniques by first filtering the video signal to 
increase the subliminal data component relative to the 
image component, then demodulating the data compo- 
nent to produce the data pattern, and then correlating 
the data pattern with the possible data patterns to iden- 
tify the particular pattern and associated data word. 

The objects, advantages and other novel features 
of the present invention are apparent from the following 
detailed description when read in conjunction with the 
appended claims and attached drawing. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF 
THE DRAWING 

Fig. 1 is a block diagram view of an encoder for 
transparently embedding data in a video signal accord- 
ing to the present invention. 

Fig. 2 is a block diagram view of a feedback model 
for determining parameter values for a human visual 



model according to the present invention. 

Fig. 3 is a block diagram view of a decoder for ex- 
tracting embedded data in the video signal according to 
the present invention. 

5 

DETAILED DESCRIPTION OF THE INVENTION 

The invisibility of embedded data in a video signal 
is achieved by several independent factors acting to- 

'0 gather. The first factor is the principle of spectral spread- 
ing from Direct-Sequence Spread Spectrum (DSSS) 
modulation of a carrier frequency. The low-bandwidth/ 
low-rate data patterns or symbols are modulated by a 
higher rate Pseudo Random Binary Sequence (PRBS), 

is spreading the power spectrum around the carrier fre- 
quency so that the time-domain waveform appears 
noise-like or lacks the spatial coherence that the eye/ 
brain is good at seeing even at low levels. Also by careful 
selection of the carrier frequency and phase it is possi- 

20 ble to further reduce the visibility of the resulting spatial 
pattern created by the PRBS modulated symbols (pat- 
terns) on a DSSS modulated carrier. 

Secondly it is also desirable to fitter the embedded 
signal both vertically and horizontally to reduce edge 

25 visibility, further enhancing it's overall invisibility. A third 
factor that is significant is to exploit visual masking by 
varying the data signal level according to the video con- 
tent. 

Another factor is the selection of a Field Alternate 

30 Carrier (FAC) frequency and phase which becomes the 
DSSS modulated carrier. The invisibility of this carrier is 
a function of it's frequency, but it's phase from one field 
to the next is very important. A first-order cancellation 
occurs by repeating the entire data pattern on the even 

3$ field (field 1 ) again on the odd field (field 2), but with an 
inverted carrier phase, causing an interlaced line pair 
cancellation. By repeating the data in each field, an 
even-line field comb fitter may be used in a decoder to 
cancel the field to field correlated video and recover the 

40 data pattern with a 3 dB advantage at a cost of a 50% 
data-rate penalty. For progressive scan video systems 
the Field Alternate Carrier frequency and data repeat 
may be replaced by a Line Alternate Carrier frequency 
and data repeat. This gives the same or better visual 

45 cancellation and has the same data rate as the inter- 
laced scan 6ystem. 

With video signals the decoder is greatly simplified 
since carrier, code and data lock are simply derived from 
the abundant timing information in sync and burst. In ad- 

50 dition to the rejection afforded by the comb filter, a band- 
pass filter is used to suppress the large carrier compo- 
nents of the program video signal. After the PRBS cor- 
relation, or de-spreading, various detection methods are 
possible. An integrate and dump matched filter driving 

& a software, bi-orthogonal M-ary scheme may be used. 
Two options with and without channel coding also may 
be used. Numerous alternatives include combining con- 
volution coding with Walsh code assignments (Ungerb- 
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oech TCM) and concatenated RS and convolution. 

Channel coding is best done in a holographic, two- 
dimensional (2D) sense by shuffling or the like for ro- 
bustness against video motion of pixel segments 
passed through the comb filter. Such 2D holographic 
shuffling of data may be extended to three-dimensional 
(3D) shuffling because video has a time sense unlike 
still pictures. This may be done by repeating some of 
the data over several frames, which could be valuable 
for video sequence identification. The data that identi- 
fies the video sequence may be given a lower energy, 
i.e., fewer pixels, than the data that identifies the partic- 
ular frame within the video sequence, since the frames 
come only once while the sequence has many frames 
in it. Data security through data encryption may always 
be applied separately from the channel coding, allowing 
the decoder design to be public or even standardized. 

Referring now to Fig. 1 an encoder 1 0 is shown hav- 
ing an analog video signal which is input to an anti-alias- 
ing, lowpass filter 12, the output of which is input to a 
sync/burst separator 14 to generate necessary timing 
signals and to an analog to digital converter (ADC) 16 
to product a digital video signal. A sample clock phase 
locked loop (PLL) 18 in response to one of the timing 
signals (subcarrier frequency fsc) from the sync/burst 
generator 1 4 provides a sample clock at four times sub- 
carrier (4fsc). Where the input signal is a digital video 
signal, the timing signals are extracted from the digital 
video signal and the lowpass fitter 12 and ADC 16 are 
not needed. The digital video signal is input to an adder 
20 where a subliminal data signal is added to the digital 
video signal. The resulting digital video signal is input to 
a digital to analog converter (D AC) 22 to produce a mod- 
ulated analog video signal which in turn is processed by 
an output lowpass reconstruction filter 24 to produce a 
final encoded analog video signal. For digital distribution 
the DAC 22 and filter 24 may be eliminated. 

A modulator circuit 30 generates the subliminal data 
signal for input to the adder 20. The modulator circuit 30 
has as inputs the vertical and horizontal timing signals 
from the sync/burst separator 14 and the desired sub- 
liminal data to be added to the digital video signal. The 
subliminal data is input to a circular shift register 32 
which has as inputs the horizontal and vertical sync sig- 
nals. The subliminal data from the circular shift register 
32 may be input to an exclusive OR gate 34 where it is 
combined with the output from a pseudo random binary 
sequence (PRBS) generator 34. The PRBS generator 
34 is reset by the vertical sync signal and clocked by a 
clock signal derived via a local PLL 36 and a divider 38. 
The output from the exclusive OR gate 35 is a series of 
+7- Vs which is then input to a mixer 40. A numerically 
controlled oscillator (NCO) 42 provides a modulator fre- 
quency fo that also is input to the mixer 40. The output 
from the mixer 40 is a modulated frequency signal which 
is the DSSS signal as modulated by the output from the 
PRBS generator 34 and the data from the circular shift 
register 32. 



To provide the Field Alternate Carrier frequency an 
inversion of the phase of either of the inputs to the mod- 
ulator 40 is performed using a divide-by-two circuit 37 
having the vertical interval timing signal as an input fol- 
5 lowed by another exclusive OR gate 39. The exclusive 
OR gate 39 has as shown the other input coupled to the 
output of the first exclusive OR gate 35, the output being 
coupled to the modulator 40. Alternatively the exclusive 
OR gate 39 may be inserted between the NCO 42 and 
10 the modulator 40, between the circular shift register 32 
and the first exclusive OR gate 35, or between the PRBS 
generator 34 and the first exclusive OR gate. For pro- 
gressive scan systems the input to the divide-by-two cir- 
cuit 37 would be the horizontal interval timing signal so 
is that the Line Alternate Carrier frequency is implement- 
ed. In either case alternate lines in each complete frame 
of the video image sequence have the same data, but 
inverted in phase, so that subtraction of alternate lines 
by a comb filter in a decoder enhances the subliminal 
20 data signal. 

The modulated frequency is input to two-dimen- 
sional, horizontal and vertical spatial frequency, soften- 
ing filters 44. The horizontal filter 44A is centered about 
the modulator frequency fo to mitigate any fast edges in 
25 the modulated frequency signal, and the vertical filter 
44B mitigates fast vertical edges (line-to-line transi- 
tions). The modulated frequency signal is the subliminal 
data signal that is input to the adder 20 via a blanking 
gate 46 so that the addition occurs only during the active 
30 video portion of the digital video signal. 

A human vision model (HVM) circuit 48 has as in- 
puts the digital video signal at the input to the adder 20. 
One practical implementation of visual masking based 
upon the human visual model is shown. A first aspect of 
the human visual model is noise tolerance in areas of 
high spatial detail, and this is controlled by a two-dimen- 
sional highpass filter 41, such as a Sobel filter, followed 
by an envelope squarer 43 and a first control amplifier 
45. A second aspect is sensitivity to noise at different 
40 local contrasts, and this is controlled by a two-dimen- 
sional lowpass filter 47 followed by a transfer offset cir- 
cuit 49 and a second control amplifier 51. The parame- 
ters A1 and A2 represented by the first and second con- 
trol amplifiers 45, 51 are adjusted for a desired minimum 
45 data pattern visibility over a range of picture material. A 
third aspect may be that of temporal masking 55 where 
noise on screen cuts or motion is temporarily less visi- 
ble. The resulting outputs from these paths, describing 
the complexity of the image rendered by the digital video 
so signal, are summed by a summer 53 to produce a gain 
signal to control a multiplier 50 to which the modulated 
frequency signal is input prior to the adder 20. Thus for 
portions of the digital video signal where the image is 
unchanging in motion and/or has low spatial detail and 
55 contrast or color/shading variations, thegain is relatively 
low; and where the image is complex due to motion and/ 
or high spatial detail, the gain is relatively high. In fact 
the amplitude of the subliminal data signal does not 
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need to be less than the noise floor in complex regions 
of the video digital signal to retain an acceptable level 
of invisibility. 

A feedback model 60 for determining the parameter 
values A1, A2 represented by the first and second con- 
trol amplifiers 45, 51 and the transfer curve of the trans- 
fer-offset circuit 49 is shown in Fig. 1 . A desired or ac- 
ceptable distortion value is pre-set and dynamically 
maintained by feedback based on the human visual 
model. An image sequence is stored in a video buffer 
62 and repeatedly output in order to compare the output 
from the human visual model 64 prior to the encoder 10 
with the image sequence derived from the output of the 
human visual model 66 following the data encoder. The 
comparison between the human visual model outputs is 
performed by means of a means absolute difference or 
mean square error algorithm 68. The resulting differ- 
ence or error is input to a comparator 69 to which also 
is input a distortion level which is adjustable to the de- 
sired level of invisibility. The parameters of the first and 
second control amplifiers 45, 51 and the transfer curve 
of the transfer-offset circuit 49 are adjusted to maintain 
comparator 69 output high. This is repeated over a 
range of exemplary image sequences. This then pro- 
duces an acceptable estimate of these parameters 
which may be fixed for all video to be encoded. The hu- 
man visual model is only as elaborate as necessary to 
match the dominant elements of the human visual sys- 
tem as it relates to the desired level of masking. 

This feedback model may also be used without the 
masking modulator 48 in an iterative manner to directly 
determine the pattern level gain signal for input to the 
multiplier 50. In other words a vector or array of gain 
signal values, maximized for each pixel, may be deter- 
mined based on the human visual model feedback, op- 
timizing the root mean square (RMS) value of the em- 
bedded pseudo-noise pattern while maintaining the de- 
sired level of distortion. 

Forward error correction may be added to the mod- 
ulator 30 by inputting the subliminal data to a forward 
error correction (FEC) circuit 52 prior to input into the 
circular shift register 32. This will provide error correc- 
tion for bit or byte errors when the data from the circular 
shift register 32 is input directly to exclusive OR gate 35. 
The FEC circuit 52 may use a bit-error correction code, 
such as Golay codes, as the forward error correction al- 
gorithm to improve the data integrity for bit errors, or a 
more sophisticated channel coding such as concate- 
nated convolution and Reed-Solomon (RS) code may 
be used, as is well known in the art. These coding tech- 
niques combined with bit shuffling provide error correc- 
tion across frames so that entire frames may be lost 
without any loss in data. For example FEC 52 may be 
designed to provide RS word-error correction where 
each frame contains a coded binary word of a fixed 
length of m data bits, allowing missing words (frames) 
to be completely recovered from the redundancy in ad- 
jacent words (frames). In this case the output from the 



circular shift register 32 may be input to an orthogonal 
pattern selector 54, such as a Walsh encoder, to pro- 
duce the subliminal data coded to select one of a set of 
orthogonal patterns input to the exclusive OR gate 35. 
s Walsh codes are one-dimensional orthogonal sequenc- 
es used in the well-known IS-95 Code Division Multiple 
Access wireless telephone standard. Also Walsh se- 
quences are the basis for a particularly well-known set 
of two-dimensional orthogonal patterns called Haad- 

10 amard patterns. Orthogonal patterns are those that, 
when correlated with all other patterns in the set, pro- 
duce a positive correlation coefficient for only one of the 
patterns, with all of the other patterns producing a zero 
correlation coefficient. Where each pattern represents 

is m bits, a total of 2^ patterns are required and the data 
from the circular shift register 32 acts as an address to 
the pattern selector 54. By using bi-orthogonal patterns 
only 2 A (m-1) patterns or sequences are needed for m 
bits. These patterns may be input directly to the mixer 

20 40 without the need of exclusive OR gate 35, but the bi- 
orthogonal patterns or Walsh sequences may not all ap- 
pear visually random, which would make them more vis- 
ible when added to the program video signal. However 
when this pattern is exclusive OR'd with the PRBS gen- 

25 erator output, the resulting light pattern at the output of 
the modulator 40 becomes much more random in ap- 
pearance and is difficult to distinguish from noise. The 
resulting modulation is a randomized bi -phase modula- 
tion spreading the spectral energy around the carrier. 

30 Circular shift register 32 allows the data or data pattern 
to be repeated on interlaced fields so that the visual line 
pair cancellation and field comb data separator in the 
decoder may be used, as described above. 

A decoder 70 is shown in Fig. 3. The input section 

35 72 is the same as for the encoder 1 0, having a lowpass 
anti-aliasing filter 74, a sync/burst separator 76 to gen- 
erate the timing signals, a PLL 78 to generate the sam- 
ple clock and an analog to digital converter 80 to pro- 
duce the digital video signal. The decoder 70 also in- 

40 eludes a Local Oscillator (LO) sectbn 82 which includes 
a PRBS generator 84 with PLL 86 and divider 88 and a 
numerically controlled oscillator (NCO) 90 that produces 
the demodulation carrier frequency fo. The outputs from 
the PRBS generator 84 and the NCO 90 are input to a 

45 mixer 91 to produce a pseudo random demodulated LO 
signal (BPSK -- Bi-Phase Shift Keyed) for a down-con- 
verter mixer 102. 

The digital video signal is input to preprocessing fil- 
ters 92 that include a bandpass filter 94 centered at the 

so modulator frequency and a high pass field-comb filter 
96 including a field delay 98 and a subtractor 1 00. These 
fitters remove much of the program video signal, en- 
hancing the detection of the subliminal embedded data 
signal. The field-comb filter 96 is particularly effective 

55 since the embedded data pattern is repeated with in- 
verse polarity on adjacent fields, doubling it's level at the 
fitter output while the program video signal that is cor- 
related on adjacent fields is removed. The filtered digital 
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video signal is input to the down-converter mixer 102 
and mixed together with the pseudo random modulated 
LO frequency signal. The resulting baseband digital sig- 
nal is input to a correlation/matched filter circuit 104. 

The correlation/matched filter circuit 104 has as in- 
puts the baseband digital signal and the horizontal and 
vertical interval timing signals. The timing signals are 
input to an orthogonal pattern generator 106, such as a 
Walsh generator, that is comparable to that in the en- 
coder 1 0. The timing signals are also input to a matched 
filter 1 08 to which also is input the baseband digital sig- 
nal. The matched filter 108 includes an integrate and 
dump circuit 110 followed by a sampler 112. If the em- 
bedded data is only as bits encoded with an FEC chan- 
nel code, as discussed previously, matched fitter 108 
output may be converted to binary data by slicer 124 
and input directly to FEC decoder 126 to output the de- 
coded and error corrected subliminal data. In the case 
where the data is coded as bi-orthogonal patterns, the 
output from the sampler 112 is input to a bi-orthogonal 
correlation receiver 114 that includes a correlation mixer 
116 for each of the patterns produced by the pattern 
generator 106 so that the sampler output is correlated 
with each pattern. The outputs from the correlation mix- 
ers 116 are integrated by respective integrators 118, the 
outputs of which are input to a maximum magnitude se- 
lector 120. Due to the orthogonality of the patterns only 
one of the outputs from the integrators 118 has a signif- 
icant magnitude that is detected by the magnitude and 
magnitude polarity selector 120 as one of the predeter- 
mined m bit patterns or data words assigned to each 
pattern. These m data words may have been FEC cod- 
ed as described previously. In this case the error cor- 
rector 1 26 is designed to correct word errors based upon 
the redundancy in other decoded data words in order to 
correct data errors from missing video frames or from 
heavily distorted embedded data patterns. The magni- 
tude selector 120 detects not only the greatest magni- 
tude, but the polarity of that magnitude to provide an 
extra bit. In other words eight patterns represent three 
bits plus one bit for the eight inverted patterns, providing 
m=4 bits. 

The subliminal data channel provided by the 
present invention may be used for stamping an identi- 
fying mark on a video sequence even if some of the vid- 
eo frames have been lost or portions of video frames 
have been cropped or distorted through lossy compres- 
sion/decompression processing. The identifying mark 
may represent an audio signal or at least a signature of 
the audio into the active video to allow a remote video 
receiver to compare the audio signature with the re- 
ceived audio to automatically advance or delay the rel- 
ative audio to video timing to maintain sound to video 
synchronization as encoded at the source. The sublim- 
inal data channel may represent modem cryptographic 
digital signature authentication in the video. It may be 
used to identify a motion test sequence, or portion there- 
of, to be captured for video quality assessment, such as 



discussed in co-pending U.S. Patent Application No. 
08/605,241 filed February 12, 1996 by Bozidar Janko 
and David K. Fibush entitled "Programmable Instrument 
for Automatic Measurement of Compressed Video 

6 Quality". The subliminal data channel also may be used 
to analyze timeliness of service, i.e., did the video get 
aired at the desired time in its entirety? 

Thus the present invention provides for transpar- 
ently encoding data into a video image by modulating a 

io particular carrier frequency, providing interlaced field 
visual masking with a unique one of 2^-1) bi-orthog- 
onal patterns, the pattern representing m bits, randomiz- 
ing the pattern with a PRBS sequence to reduce visibil- 
ity, amplitude modulating the random pattern modulated 

is carrier to further maximize embedded signal amplitude 
and minimize visibility based on a human visual model 
expbiting visual masking, and adding the resulting pat- 
tern modulated frequency signal to the video such that 
the data is recoverable using appropriate correlation 

20 techniques. 



Claims 

25 1. An apparatus for transparently encoding data within 
a video signal comprising: 



means for generating from the data to be en- 
coded and video timing signals associated with 
the video signal a randomized binary pattern; 
means for modulating the randomized binary 
pattern onto a carrier frequency to produce a 
subliminal data signal; and 
means for combining the subliminal data signal 
with the video signal to produce an output video 
signal having the data transparently encoded 
within the video signal. 



30 
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40 



The apparatus as recited in claim 1 further compris- 
ing means for alternating the phase of the sublimi- 
nal data signal on alternate lines of each frame of 
the video signal such that each pair of contiguous 
lines has the same data but inverted in phase. 



45 3. The apparatus as recited in claim 2 wherein the al- 
ternating means comprises: 



means for dividing a specific one of the video 
timing signals by two to produce a phase inver- 
sion control signal; and 

means for inverting the phase of the subliminal 
data signal on alternate lines of each frame in 
response to the phase inversion signal. 



so 



55 4. 



The apparatus as recited in claim 3 wherein for an 
interlaced version of the video signal the specific 
one of the video timing signals is a field rate signal 
and for a progressive scan version of the video sig- 
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nal the specific one of the video timing signals is a 
line rate signal. 

5. The apparatus as recited in claim 1 further compris- 
ing means for amplitude modulating the subliminal 
data signal prior to the combining means with a gain 
signal based upon a human visual model. 

6. The apparatus as recited in claims 1 , 2 or 5 further 
comprising means for softening edges in the sub- 
liminal data signal prior to the combining means. 

7. A method of transparently encoding data within a 
video signal comprising the steps of: 

generating from the data to be encoded and 
video timing signals associated with the video 
signal a randomized binary pattern; 
modulating the randomized binary pattern onto 
a carrier frequency to produce a subliminal data 
signal: and 

combining the subliminal data signal with the 
video signal to produce an output video signal 
having the data transparently encoded within 
the video signal. 

8. The method as recited in claim 7 further comprising 
the step of alternating the phase of the subliminal 
data signal on alternate lines of each frame of the 
video signal such that each pair of contiguous lines 
has the same data but inverted in phase. 

9. The method as recited in claim 8 wherein the alter- 
nating step comprises the steps of: 

dividing a specific one of the video timing sig- 
nals by two to produce a phase inversion con- 
trol signal; and 

inverting the phase of the subliminal data signal 
on alternate lines of each frame in response to 
the phase inversion signal. 

10. The method as recited in claim 9 wherein for an in- 
terlaced version of the video signal the specific one 
of the video timing signals is a field rate signal and 
for a progressive scan version of the video signal 
the specific one of the video timing signals is a line 
rate signal. 

11 . The method as recited in claim 7 further comprising 
the step of amplitude modulating the subliminal da- 
ta signal prior to the combining step with a gain sig- 
nal based upon a human visual model. 

12. The method as recited in claims 7, 8 or 11 further 
comprising the step of softening edges in the sub- 
liminal data signal prior to the combining step. 



1 3. The method as recited in claim 7 further comprising 
the step of shuffling the data two-dimensionally to 
provide channel coding. 

s 14. The method as recited in claim 7 further comprising 
the step of shuffling the data three-dimensionally to 
provide channel coding. 

16. An apparatus for transparently embedding data 
10 within a video signal comprising: 



a pattern generator having as inputs timing sig- 
nals associated with the video signal and the 
data to be embedded and producing as an out- 
put a randomized pattern signal; 
a modulator having as inputs the randomized 
pattern signal and a carrier frequency synchro- 
nized with the randomized pattern signal and 
producing as an output a subliminal data signal; 
and 

a combiner having as inputs the subliminal data 
signal and the video signal and producing as 
an output a transparently embedded data video 
signal. 



IB 



20 



2$ 



16. The apparatus as recited in claim 15 further com- 
prising a phase alternator circuit having as inputs a 
specific one of the video timing signals and a com- 
ponent of the subliminal data signal and having as 

30 an output a signal that produces inversion of the 
phase of the subliminal data signal on alternate 
lines of each frame of the video signal such that 
each pair of contiguous lines has the same data but 
inverted in phase. 

35 

17. The apparatus as recited in claim 16 wherein the 
phase alternator circuit comprises: 



a divide-by-two circuit having as an input the 
40 specific one of the video timing signals and hav- 

ing as an output a phase inversion control sig- 
nal; and 

an exclusive OR gate having as inputs the 
phase inversion control signal and the compo- 
45 nent of the subliminal data signal and having 

as an output the signal for inverting the phase 
of the subliminal data signal. 

18. The apparatus as recited in claim 17 wherein for an 
so interlaced version of the video signal the specific 

one of the video timing signals is a field rate signal 
and for a progressive scan version of the video sig- 
nal the specific one of the video timing signals is a 
line rate signal. 

55 

19. The apparatus as recited in claim 15 further com- 
prising a softening filter having as an input the sub- 
liminal data signal and producing as an output a fil- 
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tared subliminal data signal for input to the combin- 
er as the subliminal data signal. 

20. The apparatus as recited in claims 1 5, 1 6 or 1 9 fur- 
ther comprising an amplitude modulator having as 
inputs the subliminal data signal and a gain control 
signal based upon a human visual model and pro- 
ducing as an output an amplitude modulated sub- 
liminal data signal as the subliminal data signal for 
input to the combiner. 

21. The apparatus as recited in claim 15 wherein the 
pattern generator comprises: 

a data pattern generator having as inputs the 
data to be embedded and the timing signals 
and producing as an output a binary pattern; 
a pseudo random binary sequence generator 
having as an input one of the timing signals and 
producing as an output a random binary se- 
quence; 

a combination circuit having as inputs the bina- 
ry pattern and the random binary sequence and 
producing as an output the randomized binary 
pattern. 

22. The apparatus as recited in claim 21 wherein the 
data pattern generator comprises a circular shift 
register having as inputs the data and the timing sig- 
nals and producing as an output the binary pattern. 

23. The apparatus as recited in claim 22 wherein the 
data pattern generator further comprises a pattern 
encoder having the output from the circular shift 
register and the timing signals as inputs and pro- 
ducing as an output the binary pattern. 

24. The apparatus as recited in claim 23 wherein the 
data pattern generator provides a plurality of bi-or- 
thogonal binary patterns, with the one selected for 
output as the binary pattern being determined by 
the output from the circulating shift register. 

25. The apparatus as recited in claim 23 wherein the 
data pattern generator comprises a Walsh encoder 
having as input the output from the circular shift reg- 
ister and producing as an output the binary pattern 
selected from among a plurality of bi-orthogonal 
patterns. 

26. The apparatus as recited in claim 15 wherein the 
combination circuit comprises an exclusive OR gate 
having as inputs the binary pattern and the random 
binary sequence and producing as an output the 
randomized binary pattern. 

27. The apparatus as recited in claim 19 wherein the 
softening filter comprises: 



a horizontal bandpass filter centered on the car- 
rier frequency having as an input the subliminal 
data signal and producing as an output a hori- 
zontally filtered subliminal data signal; and 
s a vertical lowpass filter having as an input the 

horizontally filtered subliminal data signal and 
producing as an output the filtered subliminal 
data signal. 

10 28. The apparatus as recited in claim 20 further com- 
prising: 

a highpass human visual model filter having as 
an input the video signal and producing as an 
75 output a highpass filtered video signal repre- 

senting high spatial detail; 
a lowpass human visual model fitter having as 
an input the video signal and producing as an 
output a lowpass filtered video signal repre- 
ss senting local contrast; and 

a combining circuit having as inputs the low- 
pass and highpass filtered video signals and 
producing as an output the gain signal. 

25 29. The apparatus as recited in claim 28 further com- 
prising a temporal human visual model filter having 
as an input the video signal and producing as an 
output a temporally filtered video signal, the tempo- 
rally filtered video signal being applied as another 
30 input to the combining circuit so that the gain signal 
is a combination of the lowpass, highpass and tem- 
porally filtered video signals. 

30. The apparatus as recited in claim 20 wherein the 
35 amplitude modulator is inserted between the mod- 
ulator and the softening filter. 

31. The apparatus as recited in claim 20 wherein the 
amplitude modulator is inserted between the sof- 

40 tening filter and the combiner. 

32. The apparatus as recited in claims 1 5, 16, or 1 9 fur- 
ther comprising a forward error correction circuit 
having as an input the data to be transparently en- 

46 coded within the video signal and producing as an 
output a forward corrected data signal coupled to 
the input of the pattern generator. 

33. An apparatus for extracting transparently embed- 
so ded data from a video signal where the embedded 

data is a randomized binary pattern modulated onto 
a carrier frequency comprising: 

means for demodulating the video signal about 
55 the carrier frequency with a random binary se- 

quence modulated carrier frequency to pro- 
duce a broadband video signal; and 
means for correlating the broadband video sig- 
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nal with a plurality of binary patterns to recover 
the embedded data. 

34. The apparatus as recited in claim 33 further com- 
prising means for filtering the video signal prior to 
the demodulating means to enhance the embedded 
data with respect to the video content of the video 
signal. 

35. The apparatus as recited in claim 34 wherein the 
filtering means comprises: 

means centered on the carrier frequency for 
broadband filtering the video signal to produce 
a first filtered video signal; and 
means for comb filtering the first filtered video 
signal to produce the video signal for input to 
the demodulating means. 

36. A method of extracting transparently embedded da- 
ta from a video signal where the embedded data is 
a randomized binary pattern modulated onto a car- 
rier frequency comprising the steps of: 
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20 



sociated with the video signal for comparing the 
broadband video signal with a plurality of binary 
patterns to recover as an output the embedded 
data. 

40. The apparatus as recited in claim 39 further com- 
prising a fitter having as an input the video signal 
and producing as an output a filtered video signal 
coupled to the input of the demodulator, the filter 
enhancing the embedded data with respect to the 
video content of the video signal. 

41. The apparatus as recited in claim 40 wherein the 
filter comprises: 

a broadband filter centered on the carrier fre- 
quency having as an input the video signal and 
producing as an output a first filtered video sig- 
nal; and 

a field comb filter having as an input the first 
filtered video signal and producing as an output 
the filtered video signal coupled to the input of 
the demodulator. 



demodulating the video signal about the carrier 
frequency with a random binary sequence 
modulated carrier frequency to produce a 
broadband video signal; and 
correlating the broadband video signal with a 
plurality of binary patterns to recover the em- 
bedded data. 

37. The method as recited in claim 36 further compris- 
ing the step of filtering the video signal prior to the 
demodulating step to enhance the embedded data 
with respect to the video content of the video signal. 

38. The method as recited in claim 37 wherein the fil- 
tering step comprises the steps of: 

broadband filtering the video signal about the 
carrier frequency to produce a first filtered vid- 
eo signal; and 

comb filtering the first filtered video signal to 
produce the video signal for input to the demod- 
ulating step. 

39. An apparatus for extracting transparently embed- 
ded data from a video 6ignal where the embedded 
data is a randomized binary pattern modulated onto 
a carrier frequency comprising: 

a demodulator having as inputs a random bina- 
ry sequence modulated carrier frequency and 
the video signal and producing as an output a 
broadband video signal; and 
a correlation receiver having as inputs the 
broadband video signal and timing signals as- 



25 42. The apparatus as recited in claim 39 wherein the 
demodulator comprises: 



a pseudo random binary sequence generator 
having as an input one of the timing signals and 
providing as an output a random binary se- 
quence; 

a numerically controlled oscillator having as an 
input a control signal derived from the one tim- 
ing signal and providing as an output the carrier 
frequency; 

a first modulator having as inputs the carrier fre- 
quency and the random binary sequence and 
producing as an output a modulated carrier fre- 
quency; and 

a second modulator having as inputs the video 
signal and the modulated carrier frequency and 
producing as an output the broadband video 
signal. 
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43. The apparatus as recited in claim 39 wherein the 
correlation receiver comprises: 

a sampler having as an input the broadband 
video signal and producing as an output a sam- 
pled video signal; 

a pattern buffer having as inputs the timing sig- 
nals and providing as outputs in parallel a plu- 
rality of binary patterns; 
a plurality of correlation circuits in parallel, each 
having as inputs the sampled video signal and 
a specific one of the plurality of binary patterns 
and producing as an output a specific correla- 
tion coefficient; and 
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a magnitude comparator having as inputs the 
correlation coefficients from each correlation 
circuit and producing as an output a binary data 
word representing the sign and value of the 
greatest of the correlation coefficients, the bi- 5 
nary data word being the embedded data. 
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